Simulation devices in interventional radiology: validation pending.

MEDICAL simulation offers a tantalizing breadth and depth of potential for training and assessment in interventional radiology. It promises to provide solutions to many of the shortcomings of our traditional “apprenticeship” training. Mandatory restrictions on in-hospital work hours of resident trainees limit the time available for training and the breadth of case material to which the individual trainee is exposed. At the same time, advances in noninvasive imaging have reduced trainee exposure to invasive procedures. Thus today’s resident/trainee has limited opportunity to acquire the basic gateway skills (eg, selective diagnostic catheter angiography), upon which more advanced interventional skills are based (1–3). Medical simulators engineered for interventional radiology training have the potential to address these gaps. Simulators introduce a novel capability not only to train but also to establish objective evidence of technical competence during and after training (4–6). Although there is growing evidence for their effectiveness, few medical procedural simulations have demonstrated predictive validity. In other words, in very few instances has proficiency with a medical procedural simulator been proved to transfer to the clinical situation. This transfer of trained skills to patients has now been shown for simulations of laparoscopic surgery, colonoscopy, and anesthesia (7–9), but at the time of writing, similar evidence is still being sought for endovascular simulators. In the future, it is likely that simulations will be incorporated into certification examinations for interventional radiology (6). Although it may be intuitive that skills learned on simulators should effectively transfer to the clinical interventional radiology environment, intuition is not evidence. A great deal of work on the development and validation of interventional radiology procedural simulations must be completed before the inclusion of simulations on board, and other statutory, certification examinations can be endorsed. Ideally, the development and validation of the critical measures of performance (metrics) and test items to be used in simulators should be accomplished through a joint effort of professional societies and the certifying bodies. Only in this way will we ensure that the test instrument is compatible with the educational curriculum and that the desired competencies are being assessed. Input will be required from psychologists and experts in the subject matter, who will analyze knowledgeandtaskperformance,breaking them down into their key components (10,11). Metrics must be identified and used specifically for assessment of the learner. By design, this can be made to occur automatically (by a simulator) within the context of a simulation. The subject experts involved in test development must be appointed with complete transparency by the certifying authorities. They must faithfully represent a robust interventional radiology curriculum, with all facets of content, skill, and even geographic diversity. Ideally a single, comprehensive set of procedures, skill sets, and metrics should be defined and provided as open source for incorporation into academic and commercial simulator models. Each set of metrics based on the test items must in turn be validated for its stated purpose. The diversity of training environments represented within and across radiology societies provides an excellent opportunity for careful test validation. The use of trainees in transfer-of-training studies will show whether skills acquired through simulation indeed translate into performance in patients. Subsequent review would demonstrate whether that performance is maintained. Funding and support for this work should derive jointly from industry, the various specialty societies, and government agencies in the form of public-private partnerships. This article is also appearing in the February 2006 issue of Cardiovascular and Interventional Radiology with the title “Simulation Devices in Interventional Radiology: Caveat Emptor”.